Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC: Use default key chain for authentication to the registry #254

Closed
wants to merge 1 commit into from
Closed

RFC: Use default key chain for authentication to the registry #254

wants to merge 1 commit into from

Conversation

aholyoake-bc
Copy link

@aholyoake-bc aholyoake-bc commented Nov 11, 2020 •
edited
Loading

This should allow simple specialisations of the resource to allow for specific container registry variants without changing the existing behaviour.

For example, to use this with GCR the existing registry-image image can be used as a base layer and then we can copy in a ~/.docker/config.json and optionally a credential manager to authenticate.

Our use case:

We exclusively use google container registry and currently authenticate using the _json_key:service_account.json hack described here concourse/docker-image-resource#73 (comment).
A big drawback of this approach is that a developer can read this secret by printing it out in a task.
We want to be reasonably sure that any images that are stored in our production container registry have been built by Concourse so that we have some sort of an audit trail for what has been used to build the image. One approach to achieve this is to use the VM service account / application default credentials to authenticate. Ordinarily you would achieve this by using a credential manager and a docker configuration file.

For the oci-build-task and docker-registry resources we have a partial hack in place that does just that. i.e. a Dockerfile containing

ARG VERSION=latest
FROM vito/oci-build-task:$VERSION

COPY docker-credential-gcr /usr/local/bin/
COPY config.json /root/.docker/config.json

and a config file containing:

{
  "credHelpers": {
    "eu.gcr.io": "gcr"
  }
}

However, the check stage of the docker-registry resource does not work with this as it doesn't use docker directly. We'd like to get away from the docker-registry resource either way given that development is frozen on it and it is deprecated.

This PR is largely untested, but I would like some feedback on the approach before continuing.

@aholyoake-bc
feat(credentials): Use default key chain for auth
356a485 
This should allow simple specialisation of the image to allow for specific container registry variants.

For example, to use this with GCR the existing registry-image image can be used as a base layer and then we can copy in a ~/.docker/config.json and optionally a credential manager to authenticate.
@aholyoake-bc aholyoake-bc marked this pull request as draft November 11, 2020 14:09
@aholyoake-bc aholyoake-bc marked this pull request as ready for review November 11, 2020 14:11
@vito
Copy link
Member

vito commented Nov 17, 2020 •
edited
Loading

Thanks for bringing this up - but I don't think we should go forward with this approach, for the same reason we've been rejecting PRs to enable EC2 IAM role based auth for AWS services. concourse/concourse#3023 has a lot of info on this.

The tl;dr is that Concourse expects source: to contain credentials, because it would like to treat resources as content-addressable based on their type + source configuration so that equivalent resources across teams/pipelines may be treated as a single resource. This way versions can be shared, greatly reducing the check workload on large clusters which re-use resources between pipelines and teams. If the source: omits credentials and "magically" detects versions anyway, those versions would be shared to other teams which should not have access. (Granted, those teams would still not be able to actually fetch the bits, but in principle the versions themselves may leak sensitive information, and if we were to someday store artifacts in a blobstore based on the same content-addressible approach, this problem would become much more severe.)

We're actually working on concourse/concourse#5229 right now which will allow credentials to be fetched on a worker via extensible var source types and then fed back into source: explicitly using traditional ((vars)) syntax in a pipeline. This also has the advantage of keeping resource types simple, since they only have to implement static credential configuration. Once this feature is done it should be possible to implement a var source prototype that acquires GCR credentials by running on the appropriate worker.

Hope that makes sense - happy to answer any questions or dissect flaws in this scheme. For now you may want to just run a fork, since it'll be a little while until this is all done. 🙂

@aholyoake-bc
Copy link
Author

OK, thanks, we'll do that!

@aoldershaw aoldershaw mentioned this pull request Jul 8, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xtreme-sameer-vohra xtreme-sameer-vohra Awaiting requested review from xtreme-sameer-vohra

Assignees

@vito vito

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@aholyoake-bc @vito